Gesture-based input device for a user interface of a computer 



Field 

The present invention relates to a gesture-based input device for a user 
interface of a computer and a new interface technology enabling gesture- 
5 based navigation and interaction, searching, browsing and selection of web 
pages and multimedia content graphical user interfaces (GUIs). 

Background 

10 The most commonly used technologies for gesture-based interfaces include 
data gloves, computer vision techniques and magnetic or infrared sensors. 
Data gloves are expensive and also restrains users to a device; also, data 
gloves are not suitable for use in public spaces. Computer vision systems are 
sensitive to lighting conditions and have a comparably low update rate, though 

15 they can be quite accurate. But vision systems often force the user to adapt a 
new means of interaction (e.g. head or eye movement). Infrared and magnetic 
sensors also require the user to wear a device. EFS-based navigation 
techniques require users to undergo tedious calibration procedures. 

2 0 Moreover, gesture-based touchless input devices for user interfaces of 
computers are known using cameras for determining the position and gesture 
of, e.g., a user's hand. Examples of these technologies are described in patent 
documents DE-A-42 01 943, DE-A-199 18 072, DE-A-100 07 891, DE-A-100 
56 291, US-B-6 252 598, JP-A-09258945 and JP-A-10031551. 

25 

Moreover, from DE-A-43 12 672, a touchless cursor-control-like input device 
for computers is known which comprises proximity switches for controlling 
cursor movement and other functions. Each proximity switch functions based 
on a capacitance measurement. These switches cannot detect the position of, 



- 2 - 



for example, a user's finger within their sensitive area but can merely detect 
that within the sensitive area a user's finger is located. 

Moreover, EP-A-0 698 858 describes a touchscreen based on capacitive 
5 sensors which can reduce noise. The sensor detects the current change of the 
antenna to locate the finger position. This technology requires touching the 
surface of the screen. 

Finally, US-A-6,457,585 describes a data input device wherein the user needs 
10 to wear a data glove. This known system works on the propagated radio signal 
length for transmitting the hand gesture status from the data glove. 

Summary 

15 The invention provides a gesture-based input device for a user interface of a 
computer comprising 

two pairs of electrodes scalable for any screen size wherein the electrodes 
are arranged to capture the quasi-electrostatic field surrounding the user 
in order for the graphic user interface to provide different options or 

2 0 tasks to be selected by a user, 

a platform for supporting a user, 

an quasi-electrostatic field generator source connected to the platform 
and 

a circuitry connected to the electrodes for determining, relative to each of 

2 5 the electrodes, the position of that part of a user supported by the 

platform, e.g. a user's hand, being closest to electrodes, 
wherein the position of the part of the user in each dimension of the 
electrodes is determined based on the relation of four voltage signals of 
the circuitry, respectively, each voltage signal indicating the distance 

3 0 between the part of the user and the respective electrode limiting the 

respective dimension of the electrode array, 

whereby the position within the electrode closest to the part of the user is 
determined without any calibration of the system for the user. 
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Individual embodiments of the invention are referred to in the dependent 
claims. 

5 The system according to the invention is an alternative to the computer mouse 
or the touchscreen, because it allows the user to command and control the 
cursor via plain pointing gestures. The system can be used with all standard 
monitors and especially (large) wall projections. The interface according to the 
invention does not require users to wear or hold dedicated devices; in addition 
10 it does not need to be calibrated for different users. It is thus easy to maintain 
and protect from damage. The system is thus aimed at information systems in 
public spaces, such as information kiosks or multimedia display windows, 
which constantly have to cope with these issues. 

15 Using a quasi-electrostatic field generator, the system according to the 
invention generates a weak electric field that is then coupled to the body of the 
user. Body movement (e.g. by arm or hand) results in a change of the electric 
field, which in turn is measured and analysed by the system in order to 
measure the proximity to each of the electrodes; the exact position of the arm 

20 or hand can then be computed using special approximation techniques. An 
algorithm allows it to use the system without prior calibration. Users can 
therefore control the cursor on the screen by simple pointing gestures. Objects 
can be selected by e.g. pointing at the same object for few seconds. 

25 The technology is largely built on two areas: the hardware developed is based 
on highly sensitive quasi-electrostatic field sensor (QEFS) and an array of 
electrodes. The software was designed and implemented to analyze the user 
interaction via the sensor array and estimate the arm position from it in order 
to enable browsing and selection. This includes an algorithm for adaptive 

3 0 relative reference which compensates for omitted calibration step. 

The system according to the invention is a non-contact interface that doesn't 
require the user to calibrate the system or wear a device. He can interact with 
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the computer by pointing at the screen in a very natural and playful way. It 
provides good resolution and a high update rate (up to 1kHz) for browsing web 
pages and multimedia applications. 

5 Brief Description of the Drawings 

The invention will be described in more detail herein below referring to the 
drawings in which 

10 Fig. 1 shows an embodiment of the invention (referred to as Info-Jukebox), 
Fig. 2 shows a principle of the sensor, 

Fig. 3 shows a natural electromagnetic field around a human body, 

15 

Fig. 4 shows an enhanced human body electromagnetic field, 
Fig. 5 shows an electronic circuit model for the sensor, and 
2 0 Fig. 6 shows an electrode array for the sensor. 
Detailed Description 

The Info-Jukebox is a multimedia archive installation for public spaces (see 

2 5 Fig. 1). It presents collections of multimedia information on a multi-screen (on 

multiple screens) which can be accessed via a gesture-based interface. ECCO- 
sensors translate simple pointing gestures into screen coordinates in order to 
control the cursor. The selected information (e.g. videos) is presented on the 
navigation space as well as on a large projection to ensure that the Info- 

3 0 Jukebox can be installed in large public spaces. 



In order to ensure the basic functionality of the jukebox, two types of 
information are necessary: one is the position of the cursor, and the other is 
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selection of an item. Both parameters should be determined from the user's 
arm position. And indeed, one can do so by applying basic physics theory: It is 
well-known that capacitance C exists between two objects and C is 
proportional to 

5 




(1-1) 



where d is the distance between the two objects and K is the factor of the 
object characteristic and environment effect. 

10 

The capacitance C is a basic parameter in electronic science. By obtaining the 
value C, the distance between the two objects can be determined. Moreover, if 
two objects Al and A2 are placed at fixed positions (see Fig. 2) in a coordinate 
system, the position of the new object B can be determined by measuring the 
15 capacitance between the object pairs A1B and A2B. 

This is the basic principle underlying in the sensor of the present invention. B 
corresponds to the arm of the user. Al and A2. Once the distance is measured 
between Al and B and A2 and B, one can infer the position of the object B. By 
2 0 means of suitable software routines, one can move the cursor on the screen in 
accordance with the movement of the arm. 

It is widely known that the human body can radiate (or reflect) energy in the 
form of heat, infrared waves, electromagnetic power and others. Especially, 

2 5 the electromagnetic field is most closely related to the body, since it may be 

caused by the biochemical effects in the cells, friction of clothes, space electric 
field induction or coupled from power cable. This is why sometimes people may 
feel shocked by body electricity, especially when one takes off his/her wool 
sweater on a dry winter day. Indeed, the human body is surrounded by an 

3 0 electromagnetic field more or less all the time, as can be seen in Fig. 3. 
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Given such a field, body gestures can be captured by suitable sensors and then 
translated into computer commands. One of the most common gestures in life 
is a finger or arm pointing gesture. This kind of gesture language has been 
widely adopted in human computer interfaces based, for example, on 
5 computer vision or others. Once it is possible to detect the change of the field 
caused by body movements, one can use it to control computers in some 
ways, e.g. to navigate a screen cursor or to emulate isolated buttons on the 
keyboard. 

10 Unfortunately, the natural electromagnetic field around the human body is 
extremely weak and difficult to deal with for human computer interaction, 
whereas artificially created fields are easier to control. 

When the body is immersed in an electromagnetic field, it will become a 
15 medium or path for the field and extends to the surrounding environment as 
illustrated in Fig. 4. In fact, the human body field or energy can be enhanced in 
this way to invisibly extend the measurable size of the body. 

In the method, the enhanced field is measured by a voltage coupled solution 
20 accompanied by a custom-designed array of four or more electrodes. 

The principle of the sensor may be expressed in the model of Fig. 5. 

When the user is enhanced by the artificial field, the field around him can be 
25 regarded as a signal source represented in the model as Ui. When the user 
moves his arm close to the electrode(s), the path between user and sensor 
electrode can be described as the equivalent capacitance C in the model, via 
which the signal passes to the front processing circuit and then to the 
computer. 

30 

From Fig. 5, the output of the electrode Uo can be obtained as follows: 
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1 




(0.1) 



Rl 2 + 



{iKfCf 



where \Uo\ and \Ui\ are the amplitude of the signal, 

Rl and R2 are the resistors used, and 
/is the frequency of the signal 



Since / is in the range kHz and C is in pF unit, Rl « 



{InfCf ' 



It follows 



that, 



\Uo\ « 27fCR2\Ui\ 



(0.2) 



10 



So, the measured amplitude of Uo contains the distance information of arm 
and electrode. 

15 For different users, however, the electrical property is different. It is 
impossible to couple the same voltage strength Ui to everyone without careful 
calibration. In order to overcome this shortcoming, an array of electrodes as 
demonstrated in Fig. 6 is used. 

20 The prototype electrode array is made from a metal film accounting to the four 
electrodes. Compared to the size of the hand, it can be regarded as the line 
and point problem for capacitance which, resulting in homogeneous reception 
characteristics for the electrode array 

25 The prototype array consists of four electrodes and each has a processing 
circuit as discussed above (cf. Fig. 5). If we name the signal from electrode a 
as \Uo\ L and the signal from the electrode c as \Uo\ R one can infer 




(0.3) 



3 0 C L and C R are equivalent capacitance between the arm and the electrodes 
a, c, respectively. 
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Since capacitance is proportional to 

Coc4 (0.4) 
a 

5 

where d is the distance between the two objects and K is the factor of the 
object characteristic constant and environment effect. Thus: 

10 

d L is the distance between the user's hand and electrode a, and 
likewise for d R to electrode c. From this equation, it can be seen that the 
value of V H includes the horizontal relative distance information between the 
user's hand and the electrode. Furthermore, we can cancel out the 
15 uncontrollable effects to the environment interference and the calibration 
process become unnecessary which are generally required in most of this kind 
of application. Likewise, we can infer the vertical relative distance information 
from V v , Thus: 




d T is the distance between the user's hand and electrode b , and likewise for 
d B to electrode d . 

25 The maximum and minimal value of the V H is recorded as V ffnax and V Hmin 
respectively. 

Then the horizontal location can be defined as, 

y 

X — ,L v 

V —V 

//max w Hmin 

30 

L x is the horizontal length of the screen. 



- 9 - 

The vertical location is then, 

v 

Y= ^ .Ly 

V Vmax V V min 

5 

L Y is the vertical length of the screen. 

Since the maximal and minimal value of V H and V v , or V HmUL , V Hmin ,V Vmax , 
v vmm* are the screen boundary value for each user's hand moving range, which 
10 depends on the user habit, the dynamic determination of these boundary 
values provide the flexibility for the user hand movement. Meanwhile, V H and 
V v is less affected by distance from hand to the screen surface and this 

method also enable the user to move one's hand forward or backward freely 
before the screen while the cursor keeps still. 

15 

For the selection control, Objects can be selected by pointing at them for some 
time (between 1-2 seconds). The cursor control is enabled by raising the arm 
and terminated by dropping it. When the arm is raised, the signal strength of 
all four signals will increase, with the opposite effect when the arm is dropped. 



20 



